Characterising the Area Under the Curve Loss Function Landscape
نویسندگان
چکیده
Abstract One of the most common metrics to evaluate neural network classifiers is area under receiver operating characteristic curve (AUC). However, optimisation AUC as loss function during training not a standard procedure. Here we compare minimising cross-entropy (CE) and optimising directly. In particular, analyse landscape (LFL) approximate (appAUC) functions discover organisation this solution space. We discuss various surrogates for approximation show their differences. find that characteristics appAUC are significantly different from CE landscape. The improves testing AUC, has substantially more minima, but these minima less robust, with larger average Hessian eigenvalues. provide theoretical foundation explain results. To generalise our results, lastly an overview how LFL can help guide analysis selection.
منابع مشابه
Estimating a Bounded Normal Mean Under the LINEX Loss Function
Let X be a random variable from a normal distribution with unknown mean θ and known variance σ2. In many practical situations, θ is known in advance to lie in an interval, say [−m,m], for some m > 0. As the usual estimator of θ, i.e., X under the LINEX loss function is inadmissible, finding some competitors for X becomes worthwhile. The only study in the literature considered the problem of min...
متن کاملBoosting the Area under the ROC Curve
We show that any weak ranker that can achieve an area under the ROC curve slightly better than 1/2 (which can be achieved by random guessing) can be efficiently boosted to achieve an area under the ROC curve arbitrarily close to 1. We further show that this boosting can be performed even in the presence of independent misclassification noise, given access to a noise-tolerant weak ranker.
متن کاملEstimation of the Multivariate Normal Mean under the Extended Reflected Normal Loss Function
متن کامل
Improving the Sensitivity of Research Publication Indices Using Exponential Function and the Area under Curve
In collecting and analyzing information about recent research papers, authors, and journals with their citation and indices, we noticed that in many cases such indices fail to reflect reality and are incapable to distinguish between different authors in some cases while they clearly have different research profiles. There are many research papers that can be found in literature that discussed t...
متن کاملGeneralization Bounds for the Area Under the ROC Curve
We study generalization properties of the area under the ROC curve (AUC), a quantity that has been advocated as an evaluation criterion for the bipartite ranking problem. The AUC is a different term than the error rate used for evaluation in classification problems; consequently, existing generalization bounds for the classification error rate cannot be used to draw conclusions about the AUC. I...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Machine learning: science and technology
سال: 2022
ISSN: ['2632-2153']
DOI: https://doi.org/10.1088/2632-2153/ac49a9